Multi-band image photographing method and apparatus, and program for executing the method

ABSTRACT

The multi-band image photographing method and apparatus photograph a subject by dividing a photographing wavelength region into plural bands and obtain spectral images of the subject corresponding to the respective divided plural bands. The method and apparatus detect a sensitivity balance among the respective plural bands from photographing data of each of the spectral images obtained by preliminary photographing performed prior to main photographing, determine photographing conditions for the respective plural bands based upon the detected sensitivity balance, and perform the main photographing in accordance with the determined photographing conditions to photograph a multi-band image. The program is used to cause a computer to execute this method or part thereof.

This application is a divisional of U.S. Ser. No. 10/652,556, filed Sep.2, 2003, which claims priority from JP Application No. 2002-252887,filed Aug. 30, 2002, the entirety of each of which is incorporatedherein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a multi-band image photographing methodand apparatus, and a program for executing the method. In particular,the invention relates to a multi-band image photographing technique withwhich a multi-band image providing satisfactory noise balance to areproduced image and contributing to reproduction of a natural image isobtained.

2. Description of the Related Art

In recent years, the progress of an image pickup technique is attainingcommercialization of a multichannel photographing unit (multi-bandcamera), in particular, a photographing unit having plural channels(bands) that allow a spectral waveform of a subject to be restored witha practically sufficient accuracy. This is a camera with which a subjectis photographed in plural wavelength regions (multi-band) that allowplural kinds (four or more kinds in many cases) of light to passtherethrough, thereby obtaining plural image data (spectral image data(multi-band image data) for each band (channel)). The camera is mainlyconstructed by, for example, a CCD and a color separation filter.

In the multi-band camera, reflected light obtained by illuminating asubject with illumination light having a specified (constant) spectralintensity distribution is allowed to pass through a spectral filterhaving a specified (constant) spectral transmittance distribution whichis different for each band. The light is then focused on alight-receiving surface of an image pickup device via an optical lens ofthe camera having a specified spectral transmittance distribution,whereby a multi-band image having spectral images for respective pluralbands is obtained.

Therefore, in the conventional multi-band camera, the sensitivity ofeach band is determined based on the product of the characteristics ofspectral filters for respective plural bands and the spectralsensitivity distribution of an image pickup device.

That is, the effective exposure amount at the time of photographing withthe multi-band camera is defined by the product of filteringcharacteristics of the spectral filters, that is, spectral transmittancedistributions and a spectral intensity distribution of illuminationlight at the time of photographing. More precisely, the effectiveexposure amount is defined by including a spectral sensitivitydistribution of the image pickup device in the product, and by furtherincluding a spectral transmittance distribution of the optical lens ofthe camera in the product.

However, deviation in these spectral characteristics may cause unbalanceof the effective exposure amount for each band, and the dynamic range ofa certain band may become narrow compared with those of other bands. Asthe result, there is a problem that an S/N ratio of the band decreasesand data in the band is acquired as image data containing noise.

On the other hand, in the case where there is a deviation in thespectral distribution of a light source as exemplified by a fluorescentlamp, a deviation occurs in the intensity of the illumination lightsource in each band. As a result, image data obtained by photographingin a band having a low illumination light source intensity provides animage with lower S/N ratio than that in image data obtained byphotographing in other bands. The effective exposure amount obtained byphotographing in each band using a multi-band camera can be deemed to begiven by a function of the “spectral sensitivity of a multi-band camera”corresponding to each band and the “spectral energy distribution of alight source” corresponding to each band. The amount of light obtainedin each band and the S/N ratio has a functional relation in which theS/N ratio is improved together with the increase of the amount of light.

In this case, in a reproduced image obtained by compositing spectralimages of the respective bands, noise or unevenness in a certain colorbecomes conspicuous, that is, decrease in image quality occurs.

An exemplary method of removing such noise includes a method in which anoise suppressing parameter is changed in accordance with the noiselevel by later image processing. However, there is a problem that such amethod may cause differences in the artifact upon noise suppression ifthere are too large differences in the noise level and consequently leadto reproduction of an unnatural image.

SUMMARY OF THE INVENTION

The present invention has been devised in view of the above-mentionedconventional problems, and it is a first object of the present inventionto provide a multi-band image photographing method which is capable ofobtaining a multi-band image for natural image reproduction by allowingS/N ratios among photographing bands to fall within a predeterminedrange when obtaining the multi-band image so that a reproduced image canhave a satisfactory noise balance.

A second object of the present invention is to provide a multi-bandimage photographing apparatus capable of implementing the multi-bandimage photographing method in the first object described above.

A third object of the present invention is to provide a program forexecuting the multi-band image photographing method in the first objectdescribed above.

In order to attain the first object described above, a first aspect ofthe present invention provides a multi-band image photographing methodof photographing a subject by dividing a photographing wavelength regioninto plural bands and obtaining spectral images of the subjectcorresponding to the respective divided plural bands, comprising thesteps of detecting a sensitivity balance among the respective pluralbands from photographing data of each of the spectral imagescorresponding to the respective plural bands obtained by preliminaryphotographing which is performed prior to main photographing,determining photographing conditions for the respective plural bandsbased upon the detected sensitivity balance among the respective pluralbands, and performing the main photographing in accordance with thedetermined photographing conditions for the respective plural bands.

And, in order to attain the first object described above, the firstaspect of the present invention provides a multi-band imagephotographing method of photographing a subject by dividing aphotographing wavelength region into plural bands and obtaining spectralimages of the subject corresponding to the respective divided pluralbands, comprising the steps of preparing in advance a conversion tablethat has a proportional relation between image output values of aphotographed image and effective exposure amounts upon photographing,detecting an output value balance among the respective plural bands fromphotographing data of each of the spectral images corresponding to therespective plural bands obtained by preliminary photographing which isperformed prior to main photographing, determining photographingconditions for the respective plural bands based upon the detectedoutput value balance among the respective plural bands, performing themain photographing in accordance with the determined photographingconditions, and converting the obtained image output values into theeffective exposure amounts using the conversion table and outputting theobtained effective exposure amounts.

Further, in order to attain the first object described above, the firstaspect of the present invention provides a multi-band imagephotographing method of photographing a subject by dividing aphotographing wavelength region into plural bands and obtaining spectralimages of the subject corresponding to the respective divided pluralbands, comprising the steps of detecting an output value balance amongthe respective plural bands from photographing data of each of thespectral images corresponding to the respective plural bands obtained bypreliminary photographing which is performed prior to mainphotographing, determining photographing conditions for the respectiveplural bands based upon the detected output value balance among therespective plural bands, and performing the main photographing inaccordance with the determined photographing conditions for therespective plural bands.

Preferably, in the multi-band image photographing methods accordingly tothe first aspect of the present invention, as the photographingconditions, respective exposure times for adjusting exposure amounts aredetermined.

In order to attain the second object described above, a second aspect ofthe present invention provides a multi-band image photographingapparatus comprising photographing means for photographing a subject bydividing a photographing wavelength region into plural bands andobtaining spectral images of the subject corresponding to the respectivedivided plural bands, sensitivity balance detection means for detectinga sensitivity balance among the respective plural bands fromphotographing data for the respective plural bands obtained by thephotographing means, and photographing condition determination means fordetermining photographing conditions based upon the sensitivity balanceamong the respective plural bands detected by the sensitivity balancedetection means.

Preferably, the sensitivity balance among the respective plural bands isdetected by the sensitivity balance detection means from thephotographing data of the spectral images corresponding to therespective divided plural bands obtained by preliminary photographingwhich is performed by the photographing means prior to mainphotographing, the photographing conditions for the respective pluralbands are determined by the photographing condition determination meansbased upon the detected sensitivity balance among the respective pluralbands, and the main photographing is performed by the photographingmeans in accordance with the determined photographing conditions for therespective plural bands.

And, in order to attain the second object described above, the secondaspect of the present invention provides a multi-band imagephotographing apparatus comprising photographing means for photographinga subject by dividing a photographing wavelength region into pluralbands and obtaining spectral images of the subject corresponding to therespective divided plural bands, storage means for storing a conversiontable prepared in advance so that image output values of a photographedimage by the photographing means and effective exposure amounts uponphotographing has a proportional relation, output value balancedetection means for detecting an output value balance among therespective plural bands from photographing data of each of the spectralimages corresponding to the respective plural bands obtained bypreliminary photographing which is performed by the photographing meansprior to main photographing, photographing condition determination meansfor determining photographing conditions based upon the detected outputvalue balance among the respective plural bands, and conversion meansfor converting the image output values obtained by the photographingmeans through the main photographing based on the determinedphotographing conditions into the effective exposure amounts using theconversion table stored in the storage means.

Further, in order to attain the second object described above, thesecond aspect of the present invention provides a multi-band imagephotographing apparatus comprising photographing means for photographinga subject by dividing a photographing wavelength region into pluralbands and obtaining spectral images of the subject corresponding to therespective divided plural bands, output value balance detection meansfor detecting an output value balance among the respective plural bandsfrom photographing data of each of the spectral images corresponding tothe respective plural bands obtained by preliminary photographing whichis performed by the photographing means prior to main photographing, andphotographing condition determination means for determiningphotographing conditions based upon the detected output value balanceamong the respective plural bands.

Preferably, in the multi-band image photographing apparatusesaccordingly to the second aspect of the present invention, thephotographing condition determination means determines respectiveexposure times for adjusting exposure amounts.

Now, in order to attain the third object described above, a third aspectof the present invention provides a program for executing, at leastpartially, each of the multi-band image photographing method of theabove-described first aspect of the present invention by using acomputer.

In other words, the third aspect of the present invention provides aprogram for causing a computer to execute a method of determiningphotographing conditions for respective plural bands in order to performa multi-band image photographing method of photographing a subject bydividing a photographing wavelength region into the respective pluralbands and obtaining spectral images of the subject corresponding to therespective divided plural bands, comprising the steps of calculating asensitivity balance among the respective plural bands from photographingdata of each of the spectral images corresponding to the respectiveplural bands obtained by preliminary photographing which is performedprior to main photographing, and determining the photographingconditions for the main photographing in the respective plural bandsbased upon the calculated sensitivity balance among the respectiveplural bands.

Also, the third aspect of the present invention provides a program forcausing a computer to execute a method of determining photographingconditions for respective plural bands in order to perform a multi-bandimage photographing method of photographing a subject by dividing aphotographing wavelength region into the respective plural bands andobtaining spectral images of the subject corresponding to the respectivedivided plural bands, comprising the steps of calculating an outputvalue balance among the respective plural bands from photographing dataof each of the spectral images corresponding to the respective pluralbands obtained by preliminary photographing which is performed prior tomain photographing, determining the photographing conditions for therespective plural bands based upon the calculated output value balanceamong the respective plural bands, and converting image output valuesobtained by the main photographing under the determined photographingconditions into effective exposure amounts using a conversion tableprepared in advance so that the image output values of a photographedimage and the effective exposure amounts upon photographing has aproportional relation, thereby outputting the thus obtained effectiveexposure amounts.

Furthermore, the third aspect of the present invention provides aprogram for causing a computer to execute a method of determiningphotographing conditions for respective plural bands in order to performa multi-band image photographing method of photographing a subject bydividing a photographing wavelength region into the respective pluralbands and obtaining spectral images of the subject corresponding to therespective divided plural bands, comprising the steps of calculating anoutput value balance among the respective plural bands fromphotographing data of each of the spectral images corresponding to therespective plural bands obtained by preliminary photographing which isperformed prior to main photographing, and determining the photographingconditions for the main photographing in the respective plural bandsbased upon the calculated output value balance among the respectiveplural bands.

Preferably, in the programs according to the third aspect of the presentinvention, as the photographing conditions, exposure times for adjustingexposure amounts are determined.

Alternatively, the third aspect of the present invention may be aprogram for causing a computer that controls the multi-band imagephotographing apparatus to execute the multi-band image photographingmethod of photographing a subject by dividing a photographing wavelengthregion into plural bands and obtaining spectral images of the subjectcorresponding to the respective divided plural bands.

One of such programs comprises the steps of calculating a sensitivitybalance among the respective plural bands from photographing data ofeach of the spectral images corresponding to the respective plural bandsobtained by preliminary photographing which is performed prior to mainphotographing, determining photographing conditions for the respectiveplural bands based upon the calculated sensitivity balance among therespective plural bands, and causing the multi-band image photographingapparatus to perform the main photographing in accordance with thedetermined photographing conditions for the respective plural bands.

And, another of the programs comprises the steps of calculating anoutput value balance among the respective plural bands fromphotographing data of each of the spectral images corresponding to therespective plural bands obtained by preliminary photographing which isperformed prior to main photographing, determining photographingconditions for the respective plural bands based upon the calculatedoutput value balance among the respective plural bands, and convertingthe image output values obtained by performing the main photographing inaccordance with the determined photographing conditions using aconversion table that is prepared in advance and has a proportionalrelation between the image output values of the photographed image andthe effective exposure amounts upon photographing to output the obtainedeffective exposure amounts.

In addition, the other of the programs comprises the steps ofcalculating an output value balance among the respective plural bandsfrom photographing data of each of the spectral images corresponding tothe respective plural bands obtained by preliminary photographing whichis performed prior to main photographing, and determining thephotographing conditions for the main photographing in the respectiveplural bands based upon the calculated output value balance among therespective plural bands.

BRIEF DESCRIPTION OF THE DRAWINGS

In the accompanying drawings:

FIG. 1 is a block diagram schematically showing an embodiment of amulti-band image photographing apparatus implementing a multi-band imagephotographing method according to a first embodiment of the presentinvention;

FIG. 2 is a flowchart showing a flow of processing in the multi-bandimage photographing method according to the first embodiment of thepresent invention;

FIG. 3 is a flowchart showing an exemplary method of setting an exposuretime as a photographing condition;

FIG. 4 is a block diagram schematically showing another embodiment ofthe multi-band image photographing apparatus implementing the multi-bandimage photographing method according to a second embodiment of thepresent invention;

FIG. 5 is a flowchart showing a flow of processing in the multi-bandimage photographing method according to the second embodiment of thepresent invention;

FIGS. 6A and 6B are a front view and a side view of an experimentalillumination booth for use in Examples 2 and 3, respectively;

FIG. 7 is a block diagram schematically showing still another embodimentof the multi-band image photographing apparatus implementing themulti-band image photographing method according to a third embodiment ofthe present invention;

FIG. 8 is a flowchart showing a flow of processing in the multi-bandimage photographing method according to the third embodiment of thepresent invention;

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A multi-band image photographing method and a multi-band imagephotographing apparatus, and a program for executing this method inaccordance with the present invention will now be described in detailbelow based upon preferred embodiments shown in the accompanyingdrawings.

FIG. 1 is a block diagram schematically showing an embodiment of amulti-band image photographing apparatus with which a multi-band imagephotographing method according to a first embodiment of the presentinvention is executed.

The multi-band image photographing apparatus of this embodiment includesfirst performing preliminary photographing of a subject, determiningphotographing conditions based upon the sensitivity balance amongindividual bands obtained by the preliminary photographing, andperforming main photographing according to the determined photographingconditions.

As shown in FIG. 1, a multi-band image photographing apparatus 1 of thisembodiment includes a multi-band camera 12 for photographing a subject10 to capture multi-band image data, an inter-band sensitivity balancedetector 14 for detecting the sensitivity balance among individual bandsfrom photographing data obtained by preliminary photographing, and aphotographing condition determination device 16 for determiningphotographing conditions based upon the detected sensitivity balanceamong the respective bands.

There is no particular limitation on the multi-band camera 12 used, aslong as a plurality of spectral images corresponding to the respectivebands (a multi-band image) can be obtained by photographing a subject inplural wavelength regions (in plural bands) through which plural kindsof light (not less than four in many cases) can pass.

For example, there is a multi-band camera which includes a photographinglens and a CCD sensor for photoelectrically reading an image of focusedlight, has plural (at least four) band pass filters arranged in front ofthe photographing lens, and is adapted to photograph a subject whilesequentially changing the band pass filters from one to another. Insteadof sequentially changing the band pass filters from one to another asdescribed above, it is also preferable to use a liquid crystal tunablefilter which can electrically change the spectral transmittancedistribution.

The inter-band sensitivity balance detector 14 and the photographingcondition determination device 16 are specifically included in acomputer, and receive from the multi-band camera 12 multi-band imagedata obtained by photographing the subject 10 and performs predeterminedprocessing to be described below on the received multi-band image data.

The inter-band sensitivity balance detector 14 detects (calculates) aratio of an output value (obtained intensity) of each band (channel)obtained by photographing the subject 10 with respect to a maximumoutput value as the sensitivity balance among the respective bands.

The photographing condition determination device 16 determinesphotographing conditions based upon the sensitivity balance among therespective bands such that the sensitivity balance (the calculatedratios mentioned above) may fall within a predetermined range. Althoughnot specifically limited, the photographing condition may include amethod of adjusting photographing time (exposure time) in a lowsensitivity band such that the sensitivity balance among the respectivebands may fall within the predetermined range.

FIG. 2 is a flowchart showing a flow of processing in the multi-bandimage photographing method of this embodiment.

The operation in this embodiment will be described in accordance withthe flowchart shown in FIG. 2.

First, in step 100, the multi-band camera 12 is used to performpreliminary photographing (pre-exposure) of the subject 10 for eachband, whereby an output value of each band is obtained for example froma reference white part in a subject image. Photographing data (outputvalue of each band) obtained by the preliminary photographing is sentfrom the multi-band camera 12 to the inter-band sensitivity balancedetector 14.

Next, in step 110, the inter-band sensitivity balance detector 14detects (calculates) the sensitivity balance among the respective bands.

That is, when receiving the multi-band image data obtained by thepreliminary photographing, the inter-band sensitivity balance detector14 first compares output values of the respective bands to find amaximum value among them. Next, the inter-band sensitivity balancedetector 14 calculates a ratio between the output value of each band andthis maximum value (output value for each band/maximum value).

If this ratio (output value for each band/maximum value) is close to 1in all the bands, it can be said that the respective bands havesubstantially equal sensitivity balance. However, if this ratio takes asmall value in a band, the sensitivity of the band is low.

This embodiment aims at determining photographing conditions andperforming main photographing under the determined photographingconditions to allow the sensitivity balance for each band to besubstantially equal.

The calculated maximum value and the ratios of the output values of therespective bands (with respect to the maximum value) are sent to thephotographing condition determination device 16.

Next, in step 120, the photographing condition determination device 16determines photographing conditions for the main photographing basedupon the sensitivity balance among the respective bands detected in step110.

The photographing condition determination device 16 determines thephotographing conditions such that the sensitivity balance among therespective bands calculated in step 110 may fall within a predeterminedrange. To be more specific, the device 16 determines the photographingconditions such that the ratios of the output values of the respectivebands to the maximum value are all not less than the predetermined value(takes values close to 1).

There are various photographing conditions. An exemplary case in whichthe exposure amount adjustment is performed based on the photographingtime (exposure time) will be described below.

FIG. 3 is a flowchart showing an exemplary method of setting theexposure time as a photographing condition.

First, in step 200, the photographing condition determination device 16detects a band in which the ratios of the output values of therespective bands to the maximum value (output values of the respectivebands/maximum value) are not more than a predetermined value 1/R (whereR is an appropriate integer, e.g., R=10).

In the band in which this ratio is not more than the predetermined value1/R, the output value is smaller than the maximum value, and thesensitivity is lower than that in the band having the maximum outputvalue. Thus, for the band with low sensitivity in which the ratio is notmore than the predetermined value 1/R, the exposure time is made longerto compensate for the low sensitivity.

That is, in step 210, for the band detected as having a ratio of notmore than the predetermined value 1/R, the exposure time is multipliedby R.

Next, in step 220, the output value of the detected band is multipliedby R to compare with the maximum value again. The ratio is newlycalculated and the new ratio (new output value for the band obtained bymultiplying by R/maximum value) is compared with the predetermined value1/R.

As a result, if the ratio is still small even after the output value ismultiplied by R and the exposure time is still insufficient, the processreturns to step 210, where the exposure time (already multiplied by R)of the band is multiplied by R again.

This operation is repeated until the ratio exceeds the predeterminedvalue 1/R to offer the satisfactory sensitivity balance among the bands.

As a result, if the ratio exceeds the predetermined value, the processproceeds to step 230, where the exposure time (multiplied by R severaltimes) at that point is set as the exposure time of the band.

Referring to the flowchart shown in FIG. 2 again, when the photographingconditions are determined in step 120, the process proceeds to step 130,where the photographing condition (exposure time) determined in theprevious step is set in the multi-band camera 12, so that photographingis performed in each band under this photographing condition to obtain amulti-band image.

Note that, the exposure time may be set through the uniform adjustmentof the exposure time for all the bands based on an average determinedfrom output values of the respective bands in the entire image. In thisway, automatic exposure adjustment (AE) in the multi-band imagephotographing is attained.

In this way, according to this embodiment, in the multi-band imagephotographing, a natural image with a good S/N balance among therespective bands is capable of being photographed.

A more specific example will be described below.

EXAMPLE 1

A multi-band camera including a spectral filter unit, a CCD camera unit,and a computer in which software for controlling these units wasincorporated was used as the multi-band camera 12.

The spectral filter unit is a liquid crystal tunable filter manufacturedby CRI, Inc., Varispec Tunable Filter RS232C I/F. This is capable ofarbitrarily selecting a central wavelength in a wavelength range of 400to 720 nm and has a wavelength half-value width of 30 nm and atransmittance of 6 to 60% (depending upon the wavelength). In addition,the CCD camera is of monochrome type CA-D4-1024A PCI I/F manufactured byDALSA Corp. and is composed of 1024×1024 pixels each of which has a sizeof 12×12 (μ). Further, the personal computer is a book type PC (Windows(registered trademark) 95) C++ manufactured by PROSIDE Corp. with a CPUof 166 MHz and a RAM of 128 Mbytes.

As to photographing conditions, a metal halide lamp with subjectilluminance of 9600 lux was used as a light source and Nikomart (f=50mm, F1.4) with F2.8 was used as a lens. In addition, ultraviolet andinfrared cut filters were used to cut rays with wavelengths of 400 nm orless and 730 nm or more, respectively. In addition, a multi-band imageis composed of 16 bands obtained by dividing the wavelength range of 410to 710 nm by a wavelength interval of 20 nm.

Further, a Macbeth chart was used as the subject 10.

Under such conditions, first, preliminary photographing (pre-exposure)was performed with exposure for 25 msec in each band. Then, outputvalues of the respective bands were obtained from a reference white partin the image (white part in six gray patches of the Macbeth chart).

Since output values from a reference white chart (whose spectralreflectance is already known) cannot be generally obtained, an outputvalue of a brightest point (point where a sum of output values of therespective bands is largest) or an average value in terms of LATD in theimage may be referred to.

Next, from among the output values obtained in the above process fromthe reference white part in the respective bands, a band in which theratio of the output value (obtained intensity) to the maximum value wasa predetermined value 1/R (where R=10) was detected.

Then, the exposure time of the detected band was multiplied by R forsetting. In this case, if the ratio of the output value multiplied by Rwith respect to the maximum value was still equal to or less than thepredetermined value 1/R, this operation was repeated.

In this way, the exposure time was set and main photographing wasperformed under this condition. It can be said that setting the exposuretime ratio of the each band in this way is equivalent to auto-whitebalance (AWB) in the multi-band image photographing.

Note that, in the setting of the exposure time, it is more preferablethat the exposure time can be adjusted uniformly for all the bands byjudging an average value of output values in the respective bands in theentire image in case of setting the exposure times for the respectivebands in the above process.

As described above, in the photographing in which the ratio is set asR=10, an exposure time multiplied by ten is set in a band of 410 nmaccording to this method.

An image represented by CIELab values estimated from the photographedmulti-band image was reproduced as a hardcopy image by PICTROGRAPHY 3000manufactured by Fuji Photo Film Co., Ltd.

The image obtained had less noise in yellow/blue colors considered to bedue to a short wave region than an image obtained without setting theexposure time and thus achieved improvement of image quality.

A second embodiment of the present invention will be described below.

The embodiment to be described below refers to the case where theeffective exposure amount obtained in each band is detected inpreliminary photographing and the photographing time of each band isadjusted so that the effective exposure amount obtained in eachphotographing band falls within a predetermined range, whereby thephotographing condition is determined and main photographing isperformed.

The effective exposure amount will be first described below.

The effective exposure amount obtained in each band through thephotographing with a multi-band camera can be represented below. Morespecifically, the effective exposure amount EEVi of the ith band isrepresented by Equation (1):EEVi=α∫E(λ)·ρ(λ)·Si(λ)dλ  (1)where λ is a wavelength, Si(λ) is a spectral sensitivity of the ithband, ρ(λ) is a spectral reflectance of a subject, E(λ) is a spectraldistribution of a light source, and α is a constant of proportionality.

The effective exposure amount was represented on a time unit basis.

In the case of an achromatic color in which the spectral reflectance ofa subject does not depend on the wavelength, ρ(λ)=ρ. Therefore, Equation(1) can be written by Equation (2) below:EEVi=α·ρ·∫E(λ)·Si(λ)dλ  (2)

Further, when the reflectance of a subject is close to 1 as in a whitechart, ρ≈1. Therefore, Equation (2) can be written by Equation (3)below:EEVi≈α·∫E(λ)·Si(λ)dλ  (3)

In general, the output signal (O) from a camera has a functionalrelation with the effective exposure amount (EEV) of the camera thatreceives light and this function is very often represented by anexponential function. In other words, the output signal represented bythe equation O=f(EEV) which is a function (f(EEV)) of the effectiveexposure amount EEV is proportional to the γth power of EEV.

In particular, the output signal from a camera is converted with a tableto have an exponent γ of 1, so that the image signal acquired (outputsignal O) can be directly proportional to the effective exposure amountEEV as shown in Equation (4) below:O=f(EEV)∝EEV  (4)

As is seen from Equation (4), an image signal representing the effectiveexposure amount EEV can be obtained by subjecting the output signal Ofrom a camera to a table conversion f⁻¹ which is an inverse transform off. In other words, the relation is schematically written as follows:Output signal from camera→Table conversion f ⁻¹→Image signal

In the embodiment to be described below, such a means is provided sothat image data acquired in each band of a multi-band camera isproportional to the effective exposure amount.

The relation of Equation (4) above can also be obtained by adjusting theexponent γ to be 1 by the use of a camera having a capability to adjustthe exponent γ.

In general, the S/N ratio of an image pickup system is a function of theamount of light (effective exposure amount in this embodiment) obtainedby photographing. In this function, the S/N ratio is increased alongwith the increase of the effective exposure amount (modeling with asquare function of the effective exposure amount is also possible).

From the above, in order to obtain images with the S/N ratios of therespective bands being close to each other, the effective exposureamount obtained in each band need only be controlled to fall within aspecified range when a subject such as a reference white board whosespectral reflectance is substantially constant is photographed with amulti-band camera. Further, signal values obtained from the multi-bandcamera need only be controlled to fall within a specified range.

It can be also said that, under a light source which have substantiallythe same properties in the wavelength direction in the energy spectraldistribution, this control is consequently made to make the sensitivitybalance uniform among the bands of the camera. Under a light source suchas a fluorescent lamp whose properties in the wavelength directiongreatly changes in the energy spectral distribution, the balance withrespect to the total effective exposure amounts is controlled among thebands.

The above description was made on a time unit basis, but taking the timedimension into consideration, the effective exposure amount obtained atthe time of photographing in each band is represented as an integral ofEEVi in Equations (1), (2) and (3) over the photographing time ΔT or aproduct of EEVi and the photographing time ΔT. The effective exposureamount can be thus adjusted by the adjustment of the photographing timeΔTi of each band.

FIG. 4 is a block diagram schematically showing another embodiment ofthe multi-band image photographing apparatus implementing the multi-bandimage photographing method according to the embodiment of the presentinvention.

As shown in FIG. 4, a multi-band image photographing apparatus 101 ofthis embodiment includes a multi-band camera 112 for photographing asubject 110 to capture multi-band image data, an inter-band output valuebalance detector 114 for detecting the output value balance amongindividual bands from photographing data obtained by preliminaryphotographing, a photographing condition determination device 116 fordetermining photographing conditions based upon the detected outputvalue balance among the respective bands, a storage 118 for storing aconversion table having a proportional relation between image outputvalue and effective exposure amount, and a converter 120 for convertingthe image output value obtained by main photographing to the effectiveexposure amount using the conversion table.

The multi-band camera 112 is similar to the camera used in the firstembodiment and is not limited in any particular way.

The inter-band output value balance detector 114, the photographingcondition determination device 116, the storage 118 and the converter120 are specifically included in a computer.

The inter-band output value balance detector 114 detects ratios 1/Ri ofoutput values of the respective bands (channels) with respect to themaximum value among the output values of the respective bands obtainedby photographing reference white included in the subject 110.

For the setting, the photographing condition determination device 116sets Ri times as long as the exposure time of the channel in which theoutput value is the maximum value as the exposure time of each channelin which the detected ratio is 1/Ri.

The storage 118 stores a previously prepared conversion table which hasa proportional relation between the image output value and the effectiveexposure amount. The converter 120 uses the conversion table to convertthe image output value obtained by performing main photographing underthe photographing condition determined in the previous process into theeffective exposure amount.

The operation in this embodiment will be described below in accordancewith the flowchart shown in FIG. 5.

First, in step 300, a conversion table between image output value andeffective exposure amount which has a proportional relation between theimage output value and the effective exposure amount is prepared inadvance and stored in the storage 118.

Then, in step 310, the multi-band camera 112 is used to performpreliminary photographing of the subject 110 for each band, therebyobtaining, for example, an output value of; each band from a referencewhite part of the subject 110. The photographing data obtained by thepreliminary photographing (output value of each band) is sent from themulti-band camera 112 to the inter-band output value balance detector114.

Then, in step 320, the inter-band output value balance detector 114detects the output value balance among the respective bands. Whenreceiving a multi-band image obtained by the preliminary photographing,the inter-band output value balance detector 114 detects a band in whichthe output value from among the output values in the respective bands inthe reference white part of the subject 110 is a maximum value andcalculates a ratio 1/Ri of an output value of each band with respect tothe maximum value.

Then, in step 330, the photographing condition determination device 116determines photographing conditions of the main photographing. When theratio of the output value of each band to the band which has a maximumoutput value is 1/Ri, it is determined that the exposure time of thisband (channel) is Ri times as long as that of the band which has themaximum output value.

The thus determined exposure time is set in the multi-band camera 112and the main photographing is thus performed in next step 340.

In step 350, the image output value obtained by the main photographingis sent to the converter 120, where the image output value is convertedinto the effective exposure amount using the conversion table betweenimage output value and effective exposure amount as stored in thestorage 118.

In the processing system including the conversion table between imageoutput value and effective exposure amount as in this embodiment,control of the image output value directly leads to control of theeffective exposure amount and direct control is thus possible.

A more specific example will be described below.

EXAMPLE 2

Prior to actually starting this example, an experimental illuminationbooth 20 as shown in FIGS. 6A and 6B was prepared. FIGS. 6A and 6B are afront view and a side view of the booth 20, respectively. As shown,fluorescent lamps 22 were arranged parallel to each other in the upperpart inside of the booth 20 and a diffuser plate 24 was placed under thefluorescent lamps 22. A subject 26 was placed obliquely in the lowerpart of the booth 20 so that the subject 26 illuminated from the aboveby the fluorescent lamps 22 through the diffuser plate 24 can bephotographed from the front.

A multi-band camera similar to the one used in Example 1 including aspectral filter unit, a CCD camera unit and a computer in which softwarefor controlling these units was incorporated was used as the multi-bandcamera 112.

We now refer to the photographing conditions. For the light source usedin Example 2, eight fluorescent lamps, color rendering improved typeN-DSL from Toshiba were arranged in the illumination booth 20 as shownin FIG. 6B. The illuminance of the subject was 3000 lux.

The other photographing conditions were similar to those in Example 1 asdescribed above. A multi-band image was formed of 16 bands and a Macbethchart was used for the subject.

Photographing was performed in accordance with the flowchart shown inFIG. 5.

To be more specific, a conversion table between image output value andeffective exposure amount was prepared in advance and preliminaryphotographing was performed through exposure for 100 msec in each band,whereby output values of the respective bands were obtained fromreference white in the image (white portion of six gray patches inMacbeth Color Checker).

Since the output value obtained is not generally from a reference whitechart (with known spectral reflectance), an output value of a brightestpoint (point where a sum of output values of the respective channels islargest) or an average value in terms of LATD in the image may bereferred to.

The ratios 1/Ri of output values of the respective bands with respect toa maximum value of a band in which an output value among the outputvalues (obtained intensity) of the respective bands obtained by thepreliminary photographing of the reference white part was the maximumvalue were calculated. In the respective channels, the exposure time forthe maximum output value was multiplied by Ri as the exposure time forthe each channel, and the thus obtained value was set as thephotographing time (exposure time) of each channel and mainphotographing was performed based on the thus set exposure time.

The conversion table prepared in advance was used to convert the imageoutput value obtained by the main photographing into the effectiveexposure amount.

In this way, in the multi-band image photographing, a natural image witha good S/N balance among the respective bands can be photographed.

A third embodiment of the present invention will be described below.

This embodiment is directed to the case where the effective exposureamount is adjusted to fall within a specified range by the repetitivecontrol using a magnitude relation of output values among bands,although the physical quantity relation between image output value andeffective exposure amount is nonlinear when there is no conversion tablebetween image output value and effective exposure amount as in theabove-mentioned second embodiment.

In some cases, γ in an ordinary camera is set to be a reciprocal numberto the 2.2th power of γ in a monitor. For example, the output value Oiand the effective exposure amount EEVi have a relation as shown byEquation (5) below:Oi=255·((EEVi−0.001)/0.999)^(0.45)  (5)

Therefore, when the values among the bands are directly reflected,excessive compensation is not performed (the compensation is performedon the underexposure side).

FIG. 7 is a block diagram schematically showing still another embodimentof the multi-band image photographing apparatus implementing themulti-band image photographing method according to the third embodimentof the present invention.

As shown in FIG. 7, a multi-band image photographing apparatus 201 ofthis embodiment includes a multi-band camera 212 for photographing asubject 210 to capture multi-band image data, an inter-band output valuebalance detector 214 for detecting the output value balance amongindividual bands from photographing data obtained by preliminaryphotographing, and a photographing condition determination device 216for determining photographing conditions based upon the detected outputvalue balance among the respective bands.

The multi-band camera 212 is similar to the camera used in the firstembodiment and is not limited in any particular way.

The inter-band output value balance detector 214 and the photographingcondition determination device 216 are specifically included in acomputer.

The inter-band output value balance detector 114 detects ratios 1/Ri ofoutput values of the respective bands (channels) with respect to themaximum value among the output values of the respective bands obtainedby photographing reference white included in the subject 110.

The inter-band output value balance detector 214 detects ratios 1/Ri ofoutput values of the respective bands (channels) with respect to themaximum value among the output values of the respective bands obtainedby photographing reference white included in the subject 210.

For the setting, the photographing condition determination device 216sets Ri times as long as the exposure time of the channel in which theoutput value is the maximum value as the exposure time of each channelin which the detected ratio is 1/Ri.

The operation in this embodiment will be described below in accordancewith the flowchart shown in FIG. 8.

First, in step 500, the multi-band camera 212 is used to performpreliminary photographing of the subject 210 for each band, therebyobtaining, for example, an output value of each band from a referencewhite part of the subject 210. The photographing data obtained by thepreliminary photographing (output value of each band) is sent from themulti-band camera 112 to the inter-band output value balance detector114.

Then, in step 510, the inter-band output value balance detector 214detects the output value balance among the respective bands. Whenreceiving a multi-band image obtained by the preliminary photographing,the inter-band output value balance detector 214 detects a band in whichthe output value of each band from the reference white part of thesubject 210 is a maximum value and calculates a ratio 1/Ri of an outputvalue of each band with respect to the maximum value.

Then, in step 520, the photographing condition determination device 216determines photographing conditions of the main photographing. Sincethis embodiment has no conversion table between image output value andeffective exposure amount as described above, the photographingconditions are determined by the repetitive control. The repetitivecontrol is performed along the flow similar to that of the flowchartshown in FIG. 3 in connection with the first embodiment described above.

In other words, the ratio of the output value (obtained intensity) ofeach channel obtained from the reference white part of the subject bythe preliminary photographing with respect to the maximum value iscalculated and a channel whose calculated ratio is not more than apredetermined ratio 1/Ri is detected. The exposure time of the detectedchannel is multiplied by Ri for the setting. The preliminaryphotographing is performed again based on the exposure time obtained bythe multiplication by Ri. The exposure time is repeatedly multiplied byRi and the preliminary photographing is repeated until the output valueratio obtained is larger than the predetermined ratio 1/Ri. Thisoperation is performed until each channel fulfills this requirement. Theexposure time is thus determined.

The thus determined exposure time is set in the multi-band camera 212and the main photographing is performed based on the set exposure time.

The exposure time may be set based on the average of the output valuesof the respective bands in the entire image to enable uniform adjustmentof the exposure time for each band.

A more specific example will be described below.

EXAMPLE 3

An experimental illumination booth 20 as shown in FIGS. 6A and 6B wasalso used when actually implementing this example.

A multi-band camera similar to the one used in Example 1 including aspectral filter unit, a CCD camera unit and a computer in which softwarefor controlling these units was incorporated was used as the multi-bandcamera 212.

We now refer to the photographing conditions. The light source wassimilar to the one used in Example 2, and eight fluorescent lamps, colorrendering improved type N-DSL from Toshiba were arranged in theexperimental illumination booth 20 as shown in FIG. 6B. The illuminanceof the subject was 3000 lux.

The other photographing conditions were similar to those in Example 1 asdescribed above. A multi-band image was formed of 16 bands and a Macbethchart was used for the subject.

Photographing was first performed in the same way as described using theflowchart shown in FIG. 8.

To be more specific, preliminary photographing was first performed usingthe photographing time of 100 msec for each band to obtain an outputvalue of each band from a reference white part of an image (whiteportion of six gray patches in Macbeth Color Checker).

The ratio of the output value (obtained intensity) of each band(channel) obtained from the reference white part with respect to themaximum value is calculated and a band whose calculated ratio is notmore than a predetermined ratio 1/R is detected. The exposure time ofthe detected band is multiplied by R for the setting. The preliminaryphotographing is performed again based on the thus set exposure time.This operation is repeatedly performed until the ratio calculatedbetween the newly obtained output value and the maximum value is largerthan the predetermined ratio 1/R.

The main photographing was performed based on the thus set photographingtime.

In this example, it became possible to adjust the effective exposureamount within a specified range by the repetitive control using amagnitude relation of output values among bands, although there is noconversion table between image output value and effective exposureamount.

The image obtained in each example had less noise in yellow/blue colorsconsidered to be due to the short wave region than an image obtainedwithout setting the exposure time as described above and thus achievedimprovement of image quality. In addition, noise in red color due to thelong wave region can be also reduced.

In addition, if software for causing a computer or the like to executethe above-described multi-band image photographing method or the stepforming a part of this method, for example, exposure amountdetermination method or method for calculating the effective exposureamount is incorporated in a program, the program can be installed in acomputer for controlling a multi-band camera or any other computer toperform automatic photographing by the multi-band camera according tothe multi-band image photographing method of the invention using themulti-band camera in combination with the computer for controlling themulti-band camera or any other computer. Alternatively, the exposureamount determined or the effective exposure amount calculated by anycomputer can be used to execute the multi-band image photographingmethod of the present invention with ease.

As described above, according to the present invention, even in themulti-band image photographing, it becomes possible to perform naturalimage photographing with a good S/N balance among individual bands.

The multi-band image photographing method and apparatus, and the programfor executing this method according to the present invention have beendescribed above in detail. However, it goes without saying that thepresent invention is not limited to the above-mentioned embodiments, andvarious improvements and modifications may be made without departingfrom the gist of the present invention.

1. A multi-band image photographing method of photographing a subject bydividing a photographing wavelength region into plural bands andobtaining spectral images of said subject corresponding to respectivedivided plural bands, comprising the steps of: detecting an output valuebalance among respective plural bands from photographing data of each ofsaid spectral images corresponding to said respective plural bandsobtained by preliminary photographing which is performed prior to mainphotographing; determining photographing conditions for said respectiveplural bands based upon said detected output value balance among saidrespective plural bands; and performing said main photographing inaccordance with said determined photographing conditions for saidrespective plural bands.
 2. The multi-band image photographing methodaccording to claim 1, wherein, as said photographing conditions,respective exposure times for adjusting exposure amounts are determined.3. A multi-band image photographing apparatus comprising: photographingmeans for photographing a subject by dividing a photographing wavelengthregion into plural bands and obtaining spectral images of said subjectcorresponding to respective divided plural bands; output value balancedetection means for detecting an output value balance among respectiveplural bands from photographing data of each of said spectral imagescorresponding to said respective plural bands obtained by preliminaryphotographing which is performed by said photographing means prior tomain photographing; and photographing condition determination means fordetermining photographing conditions based upon said detected outputvalue balance among said respective plural bands.
 4. The multi-bandimage photographing apparatus according to claim 3, wherein saidphotographing condition determination means determines exposure timesfor adjusting exposure amounts.
 5. A non-transitory computer-readablemedium or memory encoding a program for causing a computer to execute amethod of determining photographing conditions for respective pluralbands in order to perform a multi-band image photographing method ofphotographing a subject by dividing a photographing wavelength regioninto the respective plural bands and obtaining spectral images of saidsubject corresponding to respective divided plural bands, comprising thesteps of: calculating an output value balance among said respectiveplural bands from photographing data of each of said spectral imagescorresponding to said respective plural bands obtained by preliminaryphotographing which is performed prior to main photographing; anddetermining the photographing conditions for said main photographing insaid respective plural bands based upon said calculated output valuebalance among said respective plural bands.
 6. The non-transitorycomputer-readable medium or memory according to claim 5, wherein, forsaid photographing conditions, exposure times for adjusting exposureamounts are determined.